Imaging apparatus and solid-state imaging device used therein

ABSTRACT

An imaging apparatus that is mounted on a vehicle that runs on a road surface includes: a light source that emits illumination light which is infrared light; a solid-state imaging device that images a subject and outputs an imaging signal indicating a light exposure amount; and a computator that computes subject information regarding the subject by using the imaging signal. The solid-state imaging device includes: first pixels that image the subject by receiving reflected light that is the illumination light reflected off the subject; and second pixels that image the subject by receiving visible light. Information indicated by an imaging signal outputted from the first pixels is information regarding a slope of the road surface, and information indicated by an imaging signal outputted from the second pixels is information regarding an appearance of the road surface.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a U.S. continuation application of PCT InternationalPatent Application Number PCT/JP2017/042990 filed on Nov. 30, 2017,claiming the benefit of priority of U.S. Provisional Patent ApplicationNo. 62/430,035 filed on Dec. 5, 2016, the entire contents of which arehereby incorporated by reference.

BACKGROUND 1. Technical Field

The present disclosure relates to an imaging apparatus that images asubject.

2. Description of the Related Art

Conventionally, an image apparatus that images a subject is known (see,for example, Japanese Unexamined Patent Application Publication No.2011-64498).

SUMMARY

In an imaging apparatus, improvement in the accuracy of measuring thedistance to a subject and/or the accuracy of detecting the subject isdesired.

Accordingly, it is an object of the present disclosure to provide animaging apparatus in which the accuracy of measuring the distance to asubject and/or the accuracy of detecting the subject can be improved ascompared with conventional imaging apparatuses, and a solid-stateimaging device used therein.

An imaging apparatus that is mounted on a vehicle that runs on a roadsurface, the imaging apparatus including: a light source that emitsillumination light which is infrared light; a solid-state imaging devicethat images a subject and outputs an imaging signal indicating a lightexposure amount; and a computator that computes subject informationregarding the subject by using the imaging signal, wherein thesolid-state imaging device includes: first pixels that image the subjectby receiving reflected light that is the illumination light reflectedoff the subject; and second pixels that image the subject by receivingvisible light, information indicated by an imaging signal outputted fromthe first pixels is information regarding a slope of the road surface,and information indicated by an imaging signal outputted from the secondpixels is information regarding an appearance of the road surface.

A solid-state imaging device used in an imaging apparatus that ismounted on a vehicle that runs on a road surface and includes a lightsource that emits illumination light which is infrared light, thesolid-state imaging device, and a computator that computes subjectinformation regarding a subject by using an imaging signal, thesolid-state imaging device including: first pixels that image thesubject by receiving reflected light that is the illumination lightreflected off the subject; and second pixels that image the subject byreceiving visible light, wherein information indicated by an imagingsignal outputted from the first pixels is information regarding a slopeof the road surface, and information indicated by an imaging signaloutputted from the second pixels is information regarding an appearanceof the road surface.

With the imaging apparatus and the solid-state imaging device configuredas described above, the accuracy of measuring the distance to a subjectand/or the accuracy of detecting the subject can be improved as comparedwith conventional imaging apparatuses.

BRIEF DESCRIPTION OF DRAWINGS

These and other objects, advantages and features of the disclosure willbecome apparent from the following description thereof taken inconjunction with the accompanying drawings that illustrate a specificembodiment of the present disclosure.

FIG. 1 is a block diagram showing a configuration of an imagingapparatus according to an embodiment;

FIG. 2 is a schematic diagram showing a pixel array included in asolid-state imaging device according to an embodiment;

FIG. 3 is a timing diagram showing the relationship between lightemission timing and exposure timing during distance measurement by usingTOF distance measuring principle;

FIG. 4 is a block diagram showing an example in which the imagingapparatus according to an embodiment is mounted on and used in avehicle;

FIG. 5A is a side view schematically showing an example of an imagingregion to be imaged by the imaging apparatus according to an embodiment;

FIG. 5B is a side view schematically showing an example of an imagingregion to be imaged by the imaging apparatus according to an embodiment;

FIG. 6 is a side view schematically showing an example of an imagingregion to be imaged by the imaging apparatus according to an embodiment;

FIG. 7A is a plan view schematically showing an example of an imagingregion to be imaged by the imaging apparatus according to an embodiment;

FIG. 7B is a plan view schematically showing an example of an imagingregion to be imaged by the imaging apparatus according to an embodiment;

FIG. 7C is a plan view schematically showing an example of an imagingregion to be imaged by the imaging apparatus according to an embodiment;

FIG. 8 is a schematic diagram showing a state in which the imagingapparatus according to an embodiment images a road surface;

FIG. 9A is a schematic diagram showing a state in which the imagingapparatus according to an embodiment images a vehicle;

FIG. 9B is a schematic diagram showing a state in which the imagingapparatus according to an embodiment images a vehicle;

FIG. 10 is a schematic diagram showing a state in which the imagingapparatus according to an embodiment images a road surface;

FIG. 11 is a schematic diagram showing an example of the relationshipbetween the output order of IR images and the output order of W images;

FIG. 12A is a schematic diagram showing an example of the relationshipbetween the output order of IR images and the output order of IRinterpolation images; and

FIG. 12B is a schematic diagram showing an example of the relationshipbetween the output order of W images and the output order of Winterpolation images.

DETAILED DESCRIPTION OF THE EMBODIMENT

An imaging apparatus according to one aspect of the present disclosureis an imaging apparatus that is mounted on a transporter, the imagingapparatus including: a light source that emits illumination light; asolid-state imaging device that images a subject and outputs an imagingsignal indicating a light exposure amount; and a computator thatcomputes subject information regarding the subject by using the imagingsignal, wherein the solid-state imaging device includes: first pixelsthat perform imaging by using reflected light that is the illuminationlight reflected off the subject; and second pixels that image thesubject, an imaging region to be imaged by the solid-state imagingdevice includes a first region that is imaged by at least the firstpixels and a second region that is imaged by the second pixels, one ofthe first region and the second region is situated around the other ofthe first region and the second region, the computator computes thesubject information based on information from the first region andinformation from the second region, and an illumination angle of theillumination light in a vertical direction of the transporter is smallerthan a viewing angle of the imaging region in the vertical direction ofthe transporter.

Also, in the imaging apparatus, an illumination angle of theillumination light in a horizontal direction of the transporter may bedifferent from a viewing angle of the imaging region in the horizontaldirection of the transporter.

Also, in the imaging apparatus, the illumination angle of theillumination light in a horizontal direction of the transporter may belarger than the illumination angle of the illumination light in thevertical direction of the transporter.

Also, in the imaging apparatus, the subject may be an object on a roadsurface on which the transporter runs, the solid-state imaging devicemay successively perform imaging, and when the object is imaged in thefirst region at a first time, and then imaged in the second region at asecond time that is time when a predetermined period of time has elapsedfrom the first time, information regarding a distance to the object atthe first time may be used to compute the subject information at thesecond time.

Also, in the imaging apparatus, the subject may be an object on a roadsurface on which the transporter runs, the solid-state imaging devicemay successively perform imaging, and when the object is imaged in thesecond region at a first time, and then imaged in the first region at asecond time that is time when a predetermined period of time has elapsedfrom the first time, information regarding an appearance of the objectat the first time may be used to compute the subject information at thesecond time.

Also, in the imaging apparatus, the solid-state imaging device maysuccessively perform imaging, and the computation performed by thecomputator may include associating information from one of the firstregion and the second region at a first time with information from theother of the first region and the second region at a second time that isdifferent from the first time.

Also, in the imaging apparatus, the information from the first regionmay be information regarding a distance to the subject, the informationfrom the second region may be information regarding an appearance of thesubject, the computation performed by the computator may includeestimating a distance to the subject in the second region, and thesubject information may include information indicating the estimateddistance to the subject in the second region.

Also, in the imaging apparatus, when the subject is an object that iscontinuously situated in the first region and the second region, acomputation result of the first region may be associated withcomputation of the subject information in the second region.

Also, in the imaging apparatus, the first region may include a first aregion where reflected light that is the illumination light reflectedoff a road surface on which the transporter runs reaches the solid-stateimaging device and a first b region where the reflected light does notreach the solid-state imaging device.

Also, in the imaging apparatus, when the subject is an object that iscontinuously situated in the first region and the second region, acomputation result of the first a region may be associated withcomputation of the subject information in a region other than the firsta region.

Also, in the imaging apparatus, the transporter may be a vehicle thatruns on a road surface, the information from the first region may beinformation regarding a slope of the road surface, the information fromthe second region may be information regarding an appearance of the roadsurface, the computation performed by the computator may includeestimating a slope of the road surface in the second region, and thesubject information may include information indicating the estimatedslope of the road surface in the second region.

Also, in the imaging apparatus, the illumination light may be infraredlight, the first pixels may receive infrared light, and the secondpixels receive may visible light.

Also, the imaging apparatus may further include a diffuser plate thatadjusts the illumination angle.

A solid-state imaging device according to one aspect of the presentdisclosure is a solid-state imaging device used in an imaging apparatusthat is mounted on a transporter and includes a light source that emitsillumination light and a computator that computes subject informationregarding a subject by using an imaging signal indicating a lightexposure amount, the solid-state imaging device being a device thatimages the subject and outputs the imaging signal, wherein thesolid-state imaging device includes: first pixels that perform imagingby using reflected light that is the illumination light reflected offthe subject; and second pixels that image the subject, an imaging regionto be imaged by the solid-state imaging device includes a first regionthat is imaged by at least the first pixels and a second region that isimaged by the second pixels, one of the first region and the secondregion is situated around the other of the first region and the secondregion, the computator computes the subject information based oninformation from the first region and information from the secondregion, and an illumination angle of the illumination light in avertical direction of the transporter is smaller than a viewing angle ofthe imaging region in the vertical direction of the transporter.

A specific example of an imaging apparatus according to one aspect ofthe present disclosure will be described with reference to the drawings.Note that the embodiment described below shows a specific example of thepresent disclosure. Accordingly, the numerical values, shapes,structural elements, the arrangement and connection of the structuralelements, steps, the order of the steps, and the like shown in thefollowing embodiment are merely examples, and therefore are not intendedto limit the scope of the present disclosure. Among the structuralelements described in the following embodiment, structural elements notrecited in any one of the independent claims are described as arbitrarystructural elements. In addition, the diagrams are schematicrepresentations, and thus are not necessarily true to scale.

Embodiment

FIG. 1 is a block diagram showing a configuration of imaging apparatus 1according to an embodiment.

As shown in FIG. 1, imaging apparatus 1 includes light source 10,solid-state imaging device 20, computator 30, controller 40, diffuserplate 50, lens 60, and band-pass filter 70.

Light source 10 emits illumination light. More specifically, lightsource 10 emits illumination light that illuminates a subject at atiming indicated by a light emission signal generated by controller 40.

Light source 10 includes, for example, a capacitor, a driving circuit,and a light emitting element, and emits light by driving the lightemitting element by using electric energy stored in the capacitor. Thelight emitting element is implemented by, for example, a laser diode, alight emitting diode, or the like. Light source 10 may include one lightemitting element, or may include a plurality of light emitting elementsfor different purposes.

The following description will be given assuming that the light emittingelement is, for example, a laser diode that emits near infrared light, alight emitting diode that emits near infrared light, or the like, andthe illumination light emitted by light source 10 is near infraredlight. However, the illumination light emitted by light source 10 is notnecessarily limited to near infrared light. The illumination lightemitted by light source 10 may be, for example, infrared light at afrequency band outside the near infrared frequency band.

Solid-state imaging device 20 images a subject and outputs an imagingsignal that indicates the amount of light exposure (light exposureamount). More specifically, solid-state imaging device 20 performsexposure at a timing indicated by an exposure signal generated bycontroller 40, and outputs the imaging signal that indicates the lightexposure amount.

Solid-state imaging device 20 includes a pixel array in which firstpixels that perform imaging by using reflected light that is theillumination light reflected off a subject and second pixels that imagethe subject are arranged in an array. Solid-state imaging device 20 mayoptionally include, for example, a cover glass, a logic function such asan AD converter, and the like.

As with the illumination light, the following description will be givenassuming that the reflected light is near infrared light, but thereflected light is not necessarily limited to near infrared light aslong as it is the illumination light reflected off a subject.

FIG. 2 is a schematic diagram showing pixel array 2 included insolid-state imaging device 20.

As shown in FIG. 2, pixel array 2 is configured such that first pixels21 (IR pixels) that perform imaging by using reflected light that is theillumination light reflected off a subject and second pixels 22 (Wpixels) that image the subject are arranged in an array so as toalternate in each column.

Also, in FIG. 2, in pixel array 2, second pixels 22 and first pixels 21are arranged so as to be adjacent in the row direction, and secondpixels 22 and first pixels 21 are placed in alternate rows to form astripe pattern. However, the pixel arrangement is not limited thereto.Second pixels 22 and first pixels 21 may be arranged in every pluralityof rows (for example, every other two rows). That is, first rows in eachof which second pixels 22 are adjacent in the row direction and secondrows in each of which first pixels 21 are adjacent in the row directionmay be alternately arranged every M rows (where M is a natural number).Furthermore, the first rows in each of which second pixels 22 areadjacent in the row direction and the second rows in each of which firstpixels 21 are adjacent in the row direction may be arranged differently(i.e., the first rows may be arranged every N rows and the second rowsmay be arranged in every L rows (where N and L are different naturalnumbers)).

First pixels 21 are implemented by, for example, near infrared lightpixels that are sensitive to near infrared light that is the reflectedlight. Second pixels 22 are implemented by, for example, visible lightpixels that are sensitive to visible light.

The near infrared light pixels are each composed of, for example, anoptical filter that allows only near infrared light to passtherethrough, a microlens, a light receiving element that serves as aphotoelectric converter, a storage that stores electric chargesgenerated by the light receiving element, and the like. Likewise, thevisible light pixels are each composed of, for example, an opticalfilter that allows only visible light to pass therethrough, a microlens,a light receiving element that serves as a photoelectric converter, astorage that stores electric charges converted by the light receivingelement, and the like. The optical filter included in each visible lightpixel may be configured to allow both visible light and near infraredlight to pass therethrough, or may be configured to allow only light ina specific wavelength band of visible light such as red (R), green (G),or blue (B) to pass therethrough.

Again, referring back to FIG. 1, imaging apparatus 1 will be furtherdescribed.

Computator 30 computes subject information regarding the subject byusing the imaging signal output from solid-state imaging device 20.

Computator 30 is configured by using, for example, a computationprocessing device such as a microcomputer. The microcomputer includes aprocessor (microprocessor), a memory, and the like, and generates alight emission signal and an exposure signal as a result of a drivingprogram stored in the memory being executed by the processor. Ascomputator 30, an FPGA, an ISP, or the like may be used. Also,computator 30 may be configured by using one hardware component or aplurality of hardware components.

Computator 30 calculates the distance to the subject based on, forexample, TOF distance measuring principle that is performed by using theimaging signals output from first pixels 21 of solid-state imagingdevice 20.

Hereinafter, the calculation of the distance to the subject by using theTOF distance measuring principle performed by computator 30 will bedescribed with reference to the drawings.

FIG. 3 is a timing diagram showing the relationship between the lightemission timing of the light emitting element of light source 10 and theexposure timing of exposure to first pixels 21 of solid-state imagingdevice 20 when computator 30 calculates the distance to the subject byusing the TOF distance measuring principle.

In FIG. 3, Tp represents a light emission period during which the lightemitting element of light source 10 emits illumination light. Tdrepresents a delay time from when the light emitting element of lightsource 10 emits illumination light to when reflected light that is theillumination light reflected off the subject returns to solid-stateimaging device 20. First exposure period is the same timing as the lightemission period during which light source 10 emits illumination light,and second exposure period is the time from the end of the firstexposure period to the end of elapse of a light emission period Tp.

In FIG. 3, q1 represents the total amount of exposure of reflected lightto first pixels 21 of solid-state imaging device 20 during the firstexposure period, and q2 represents the total amount of exposure ofreflected light to first pixels 21 of solid-state imaging device 20during the second exposure period.

As a result of the emission of illumination light by the light emittingelement of light source 10 and the light exposure to first pixels 21 bysolid-state imaging device 20 being performed at the timing shown inFIG. 3, distance d to the subject can be represented by Equation 1 givenbelow, where c represents light velocity.

d=c×Tp/2×q1/(q1+q2)  Equation 1

Accordingly, with Equation 1, computator 30 can calculate the distanceto the subject by using the imaging signals output from first pixels 21of solid-state imaging device 20.

Again, referring back to FIG. 1, imaging apparatus 1 will be furtherdescribed.

Computator 30 performs detection of the subject and calculation of thedistance to the subject by using, for example, imaging signals outputfrom second pixels 22 of solid-state imaging device 20.

That is, computator 30 makes a comparison between a first visible lightimage imaged by a plurality of second pixels 22 of solid-state imagingdevice 20 at a first time and a second visible light image imaged by aplurality of second pixels 22 of solid-state imaging device 20 at asecond time, and performs detection of the subject and calculation ofthe distance to the subject based on the difference between the firstand second visible light images. Here, the detection of the subject maybe performed by, for example, distinguishing the shape of the subjectbased on pattern recognition by edge detection of feature points of thesubject. Also, the calculation of the distance to the subject may beperformed using world coordinate conversion.

Other examples of computation performed by computator 30 will bedescribed later.

Controller 40 generates a light emission signal that indicates thetiming of light emission and an exposure signal that indicates thetiming of exposure. Then, controller 40 outputs the generated lightemission signal to light source 10, and the generated exposure signal tosolid-state imaging device 20.

Controller 40 may cause imaging apparatus 1 to perform continuousimaging at a predetermined frame rate by, for example, generating andoutputting the light emission signal so as to cause light source 10 toemit light on a predetermined cycle and generating and outputting theexposure signal so as to cause solid-state imaging device 20 to performexposure on a predetermined cycle.

Controller 40 is configured by using, for example, a computationprocessing device such as a microcomputer. The microcomputer includes aprocessor (microprocessor), a memory, and the like, and generates alight emission signal and an exposure signal as a result of a drivingprogram stored in the memory being executed by the processor. Ascontroller 40, an FPGA, an ISP, or the like may be used. Also,controller 40 may be configured by using one hardware component or aplurality of hardware components.

Diffuser plate 50 adjusts the angle of illumination light.

Lens 60 is an optical lens that condenses external light enteringimaging apparatus 1 on the surface of pixel array 2 of solid-stateimaging device 20.

Band-pass filter 70 is an optical filter that allows near infrared lightthat is the reflected light and visible light to pass therethrough.

Imaging apparatus 1 configured as described above is mounted on and usedin a transporter. The following description will be given assuming thatimaging apparatus 1 is mounted on and used in a vehicle that runs on aroad surface. However, the transporter on which imaging apparatus 1 ismounted is not necessarily limited to a vehicle. Imaging apparatus 1 maybe mounted on and used in a transporter other than a vehicle such as,for example, a motorcycle, a boat, or an airplane.

Although the accuracy of the distance to the subject decreases ascompared with that calculated by using the TOF distance measuringprinciple described above, computator 30 may calculate the distance tothe subject without using the imaging signals from first pixels 21 ofsolid-state imaging device 20.

FIG. 4 is a block diagram showing an example in which imaging apparatus1 is mounted on and used in vehicle 100.

As shown in FIG. 4, imaging apparatus 1 is used by being connected to,for example, ADAS (Advanced Driver Assistance System)/AD-ECU (AutomatedDriving-Engine Control Unit) 110 mounted on vehicle 100.

ADAS/AD-ECU 110 is a system that is mounted on vehicle 100 and performsautomatic drive control on vehicle 100 by utilizing the signals fromimaging apparatus 1 and sensors 12A to 12C, and includes locator 111that locates the position of the vehicle, controller 112 that controls abrake, a steering wheel, an engine, and the like, and other components.

Imaging apparatus 1 may be mounted at any position on vehicle 100. Forexample, imaging apparatus 1 may be mounted at the center of the frontsurface of vehicle 100, or in other words, at the center between twoheadlights, and an area in the front direction of vehicle 100 can bedefined as an imaging region to be imaged by imaging apparatus 1.

FIG. 5A is a side view schematically showing an example of an imagingregion to be imaged by imaging apparatus 1 in the case of an externalenvironment where the imaging distance with visible light is longer thanthe imaging distance with reflected light such as the daytime of a sunnyday when imaging apparatus 1 is mounted at the center of the frontsurface of vehicle 100 so as to monitor an area in the front directionof vehicle 100.

As shown in FIG. 5A, first region 110 is a region that is imaged by atleast first pixels 21. That is, first region 110 is a region where theillumination light emitted from light source 10 is reflected off asubject, and the reflected light can reach solid-state imaging device20.

In FIG. 5A, second region 120 is a region that is imaged by secondpixels 22. That is, second region 120 is a region where ambient lightthat is visible light can reach solid-state imaging device 20.

In the example shown in FIG. 5A, the illumination angle of illuminationlight in the vertical direction of vehicle 100 is appropriately adjustedby diffuser plate 50 so as to be smaller than the viewing angle of theimaging region in the vertical direction of vehicle 100. For example,the illumination angle of illumination light in the vertical directionof vehicle 100 may be set to about 20 degrees, and the viewing angle ofthe imaging region in the vertical direction of vehicle 100 may be setto about 30 degrees.

As described above, by adjusting the illumination range of illuminationlight to be smaller than the viewing angle of the imaging region, theillumination light emitted by light source 10 that is finite energy canbe concentratedly directed to a particular target range. As a result, inthe target range, the distance to which reflected light travels can beextended as compared with the case where the illumination light is notconcentratedly directed.

As shown in FIG. 5A, in the case of an external environment where theimaging distance with visible light is longer than the imaging distancewith reflected light, second region 120 is situated farther away fromvehicle 100 than first region 110. That is, in the case of such anexternal environment, second region 120 is situated around first region110 that is the other region.

Also, as shown in FIG. 5A, first region 110 includes first a region 110a and first b region 110 b.

First a region 110 a is a region of first region 110 where reflectedlight from road surface 130 (reference surface) and reflected light fromthe subject above the elevation angle of road surface 130 can reachsolid-state imaging device 20.

First b region 110 b is a region of first region 110 where reflectedlight from the subject above the elevation angle of road surface 130 canreach solid-state imaging device 20, but reflected light from the roadsurface does not reach solid-state imaging device 20.

Also, as shown in FIG. 5A, second region 120 includes second a region120 a and second b region 120 b.

Second a region 120 a is a region of second region 120 that is situatedabove the elevation angle of the interface between first a region 110 aand first b region 110 b.

Second b region 120 b is a region of second region 120 that is situatedbelow the elevation angle of the interface between first a region 110 aand first b region 110 b.

FIG. 5B is a side view schematically showing an example of an imagingregion to be imaged by imaging apparatus 1 in the case of an externalenvironment where the imaging distance with reflected light is longerthan the imaging distance with visible light (for example, in the casewhere the external environment is nighttime, a rainy day, a dense fog,or the like) when imaging apparatus 1 is mounted at the center of thefront surface of vehicle 100 so as to monitor an area in the frontdirection of vehicle 100.

As shown in FIG. 5B, in the case of an external environment where theimaging distance with reflected light is longer than the imagingdistance with visible light, first region 110 is situated farther awayfrom vehicle 100 than second region 120. That is, in the case of such anexternal environment, first region 110 is situated around second region120 that is the other region.

As described with reference to FIGS. 5A and 5B, the range of secondregion 120 may vary due to an external factor (for example, externalenvironment). Also, likewise, the range of first region 110 may vary dueto an external factor (for example, external environment).

The application of imaging apparatus 1 is not necessarily limited tomonitoring the area in the front direction of vehicle 100 as shown inFIGS. 5A and 5B. For example, imaging apparatus 1 may be used to monitoran area in the rear direction of vehicle 100. That is, imaging apparatus1 may be mounted, for example, at the center of the rear surface of thevehicle, or in other words, at the center between two taillights, and anarea in the rear direction of vehicle 100 can be defined as an imagingregion to be imaged by imaging apparatus 1.

FIG. 6 is a side view schematically showing an example of an imagingregion to be imaged by imaging apparatus 1 in the case of an externalenvironment where the imaging distance with visible light is longer thanthe imaging distance with reflected light such as the daytime of a sunnyday when imaging apparatus 1 is mounted at the center of the rearsurface of vehicle 100 so as to monitor an area in the rear direction ofvehicle 100.

FIG. 7A is a plan view schematically showing an example of an imagingregion to be imaged by imaging apparatus 1 in the case of an externalenvironment where the imaging distance with visible light is longer thanthe imaging distance with reflected light (for example, in the casewhere the external environment is the daytime of a sunny day, or thelike) when imaging apparatus 1 is mounted at the center of the frontsurface of vehicle 100 so as to monitor an area in the front directionof vehicle 100.

In the example shown in FIG. 7A, the angle of first region 110 in thehorizontal direction of the vehicle is substantially equal to the angleof second region 120 in the horizontal direction of the vehicle. Thatis, the illumination angle of illumination light in the horizontaldirection of vehicle 100 is appropriately adjusted by diffuser plate 50so as to be substantially equal to the viewing angle of the imagingregion in the horizontal direction of vehicle 100. For example, theillumination angle of illumination light in the horizontal direction ofvehicle 100 may be set to about 90 degrees, and the viewing angle of theimaging region in the horizontal direction of vehicle 100 may be set toabout 90 degrees.

As described above, by adjusting the illumination angle of illuminationlight in the horizontal direction of vehicle 100 so as to besubstantially equal to the viewing angle of the imaging region in thehorizontal direction of vehicle 100, a subject in the full viewing anglein the horizontal direction of the vehicle can be imaged by first pixels21.

The illumination angle of illumination light in the horizontal directionof vehicle 100 may be different from the viewing angle of the imagingregion in the horizontal direction of vehicle 100. For example, theillumination angle of illumination light in the horizontal direction ofvehicle 100 may be set larger than the viewing angle of the imagingregion in the horizontal direction of vehicle 100 such that the subjectin the full viewing angle in the horizontal direction of the vehicle canbe imaged more reliably by first pixels 21. For example, in the casewhere the viewing angle of the imaging region in the horizontaldirection of vehicle 100 is set to about 90 degrees, the illuminationangle of illumination light in the horizontal direction of vehicle 100may be set to about 92 degrees.

Furthermore, imaging apparatus 1 may be used to monitor an area in adirection oblique to vehicle 100, or may be used to monitor a side areaof vehicle 100.

FIG. 7A may be used as one of a plurality of sensing apparatuses forperforming emergency brake control on vehicle 100.

Furthermore, FIG. 7A may be used as a sensing apparatus that measures alonger distance as compared with FIGS. 7B and 7C, which will bedescribed later.

FIG. 7B is a plan view schematically showing an example of an imagingregion to be imaged by imaging apparatus 1 in the case of an externalenvironment where the imaging distance with visible light is longer thanthe imaging distance with reflected light (for example, in the casewhere the external environment is the daytime of a sunny day or thelike) when imaging apparatus 1 is mounted near a headlight of vehicle100 so as to monitor an area in a direction oblique to vehicle 100.

FIG. 7C is a plan view schematically showing an example of an imagingregion to be imaged by imaging apparatus 1 in the case of an externalenvironment where the imaging distance with visible light is longer thanthe imaging distance with reflected light (for example, in the casewhere the external environment is the daytime of a sunny day) whenimaging apparatus 1 is mounted on the side surface of a side mirror ofvehicle 100 so as to monitor a side area of vehicle 100.

FIGS. 7B and 7C may be used as one of a plurality of sensing apparatusesfor performing automatic drive control on vehicle 100.

Furthermore, FIGS. 7B and 7C may be used as a sensing apparatus thatmeasures a distance that is shorter than the distance measured by FIG.7A, or an intermediate distance.

Particularly when imaging apparatus 1 is used to monitor an area in adirection oblique to vehicle 100 or a side area of vehicle 100, theillumination angle of illumination light in the horizontal direction ofvehicle 100 may be substantially equal to the viewing angle of theimaging region in the horizontal direction of vehicle 100.

FIG. 8 is a schematic diagram showing an example of a state in whichimaging apparatus 1 images road surface 130 when first region 110 andsecond region 120 are in the relationship shown in FIGS. 5A and 7A, thediagram being viewed from imaging apparatus 1.

As shown in FIG. 8, in the vertical direction of vehicle 100, secondregion 120 is situated so as to extend over an area from the upper endto the lower end of the viewing angle of the captured image, but firstregion 110 is situated so as to extend over a limited area from belowthe upper end to above the lower end of the viewing angle of thecaptured image.

On the other hand, as shown in FIG. 8, in the horizontal direction ofvehicle 100, first region 110 and second region 120 are both situated soas to extend over an area from the left end to the right end of thecaptured image.

In FIG. 8, second c region 120 c is a region that is within second aregion 120 a but outside first a region 110 a, and where reflected lightmay reach solid-state imaging device 20, but reflected light does notreach solid-state imaging device 20 in a stable manner. For this reason,computator 30 performs computation without using the imaging signalsfrom first pixels 21 corresponding to second c region 120 c.

Also, in FIG. 8, second d region 120 d is a region that is within secondb region 120 b but outside first b region 110 b, and where reflectedlight may reach solid-state imaging device 20, but reflected light doesnot reach solid-state imaging device 20 in a stable manner. For thisreason, as with second c region 120 c, computator 30 performscomputation without using the imaging signals from first pixels 21corresponding to second d region 120 d.

FIGS. 9A and 9B are schematic diagrams each showing an example of astate in which imaging apparatus 1 images vehicle 200 running on roadsurface 130 at a first time and a second time that is later than thefirst time when first region 110 and second region 120 are in therelationship shown in FIGS. 5A and 7A, the diagram being viewed fromimaging apparatus 1.

In FIGS. 9A and 9B, vehicle 200 a and vehicle 200 c are vehicles 200imaged at the first time, and vehicle 200 b and vehicle 200 d arevehicles 200 imaged at the second time that is later than the firsttime.

The examples shown in FIGS. 9A and 9B illustrate examples correspondingto scenes in which vehicle 100 approaches vehicle 200 during a periodfrom the first time to the second time. However, the distance to vehicle200 is different between the example shown in FIG. 9A and the exampleshown in FIG. 9B.

In the example shown in FIG. 9A, vehicle 200 a is within first region110, and is also within second region 120.

For this reason, computator 30 detects vehicle 200 a, which is a subjectto be imaged, based on the information from second region 120, or inother words, the imaging signals from second pixels 22 obtained at thefirst time, and also calculates the distance to vehicle 200 a, which isthe subject, with a relatively high degree of accuracy by using the TOFdistance measuring principle based on the information from first region110, or in other words, the imaging signals from first pixels 21obtained at the first time.

On the other hand, in the example shown in FIG. 9A, vehicle 200 b isoutside first region 110, but is within second region 120.

For this reason, computator 30 detects vehicle 200 b, which is a subjectto be imaged, based on the information from second region 120, or inother words, the imaging signals from second pixels 22 obtained at thesecond time, but does not perform calculation of the distance to vehicle200 b, which is the subject, by using the TOF distance measuringprinciple based on the information from first region 110, or in otherwords, the imaging signals from first pixels 21. Instead, computator 30calculates the distance to vehicle 200 b, which is the subject, based onthe information from second region 120, or in other words, the imagingsignals from second pixels 22, without using the TOF distance measuringprinciple.

Then, computator 30 associates the information from first region 110obtained at the first time and the information from second region 120obtained at the second time with each other. More specifically,computator 30 compares the result of detection of vehicle 200 aperformed at the first time with the result of detection of vehicle 200b performed at the second time. If it is determined that vehicle 200 aand vehicle 200 b are the same vehicle 200, computator 30 associates(correlates) information indicating the distance to vehicle 200 acalculated by using the TOF distance measuring principle at the firsttime and the result of detection of vehicle 200 b performed at thesecond time with each other.

Then, computator 30 estimates the distance to the subject in the secondregion. More specifically, computator 30 estimates the distance tovehicle 200 b at the second time based on information indicating thedistance to vehicle 200 a calculated by using the TOF distance measuringprinciple at the first time, the result of detection of vehicle 200 bperformed at the second time, and information indicating the distance tovehicle 200 b calculated at the second time without using the TOFdistance measuring principle.

By doing so, computator 30 can estimate the distance to vehicle 200 b atthe second time with a higher degree of accuracy as compared with thecase where calculation is performed based on only the imaging signalsfrom second pixels 22 obtained at the second time.

In the example shown in FIG. 9B, vehicle 200 c is outside first region110, but is within second region 120.

For this reason, computator 30 performs detection of vehicle 200 c,which is a subject to be imaged, and calculation of the distance tovehicle 200 c, which is the subject, without using the TOF distancemeasuring principle, based on the information from second region 120, orin other words, the imaging signals from second pixels 22 obtained atthe first time.

That is, in the example shown in FIG. 9A, the subject is an object(vehicle 200 a or vehicle 200 b) on the road surface on which thetransporter runs. Solid-state imaging device 20 successively performsimaging. When the object is imaged in first region 110 at a first time,and then imaged in second region 120 at a second time that is time whena predetermined period of time has elapsed from the first time,information regarding the distance to the object (vehicle 200 a) at thefirst time is used to compute information regarding the subject (vehicle200 b) at the second time.

On the other hand, in the example shown in FIG. 9B, vehicle 200 d iswithin first region 110, and is also within second region 120.

For this reason, computator 30 detects vehicle 200 d, which is a subjectto be imaged, based on the information from second region 120, or inother words, the imaging signals from second pixels 22 obtained at thesecond time, and also calculates the distance to vehicle 200 d, which isthe subject, with a relatively high degree of accuracy by using the TOFdistance measuring principle based on the information from first region110, or in other words, the imaging signals from first pixels 21obtained at the second time. In this regard, computator 30 also performsthe detection and the calculation based on the result of detection ofvehicle 200 c at the first time.

That is, computator 30 performs the detection and the calculation basedon, for example, the result of detection of vehicle 200 c at the firsttime by limiting the search range to a partial region.

By doing so, computator 30 can perform the detection and the calculationin a shorter time than that when the detection and the calculation areperformed without using the result of detection of vehicle 200 c.

Also, computator 30 detects vehicle 200 d at the second time based on,for example, the result of detection of vehicle 200 c at the first timeand the imaging signals from second pixels 22 at the second time.

By doing so, computator 30 can detect vehicle 200 d at the second timewith a higher degree of accuracy than that when the detection isperformed without using the result of detection of vehicle 200 c at thefirst time.

That is, in the example shown in FIG. 9B, the subject is an object(vehicle 200 c or vehicle 200 d) on the road surface on which thetransporter runs. Solid-state imaging device 20 successively performsimaging. When the object is imaged in second region 120 at a first time,and then imaged in first region 110 at a second time that is time when apredetermined period of time has elapsed from the first time,information regarding the appearance of the object (vehicle 200 c) atthe first time is used to compute information regarding the subject(vehicle 200 d) at the second time.

As described above with reference to FIGS. 9A and 9B, when the positionof the subject (vehicles 200 a to 200 d) changes (1) from first region110 to second region 120, or (2) from second region 120 to first region110 along with the elapse of time from the first time to the secondtime, the information from one of first region 110 and second region 120at the first time is associated with the information from the other offirst region 110 and second region 120 at the second time that isdifferent from the first time, as a result of which it is possible toperform sensing (distance measurement, detection, and the like) of thesubject at the second time in a short time and/or with a high degree ofaccuracy.

FIG. 10 is a schematic diagram showing an example of a state in whichimaging apparatus 1 images road surface 130 when first region 110 andsecond region 120 are in the relationship shown in FIGS. 5A and 7A, thediagram being viewed from imaging apparatus 1. That is, the exampleshown in FIG. 10 corresponds to the same scene as that of the exampleshown in FIG. 8. However, for the sake of better understanding of theoperations of computator 30, the illustration of FIG. 10 is partiallyshown in a different manner from that of FIG. 8.

In the example shown in FIG. 10, roadside boundary 140 that indicatesthe road sides of road surface 130 includes roadside boundary region 140a and roadside boundary region 140 b that are included in first a region110 a, and roadside boundary regions 140 c, 140 d, 140 e, and 140 f thatare not included in first a region 110 a.

As described above, roadside boundary region 140 a and roadside boundaryregion 140 b are included in first a region 110 a. For this reason,computator 30 detects road surface 130 (or in other words, roadsideregion) in roadside boundary region 140 a and roadside boundary region140 b based on the information from second region 120, or in otherwords, the imaging signals from second pixels 22, and also calculatesthe distance to road surface 130 (or in other words, roadside region) inroadside boundary region 140 a and roadside boundary region 140 b with arelatively high degree of accuracy by using the TOF distance measuringprinciple based on the information from first a region 110 a, or inother words, the imaging signals from first pixels 21.

By doing so, computator 30 can calculate the appearance and the slope ofroad surface 130 (or in other words, roadside region) in roadsideboundary region 140 a and roadside boundary region 140 b.

On the other hand, as described above, roadside boundary regions 140 c,140 d, 140 e, and 140 f are not included in first a region 110 a. Forthis reason, computator 30 performs detection of road surface 130 (or inother words, roadside region) in roadside boundary regions 140 c, 140 d,140 e, and 140 f based on the information from second region 120, or inother words, the imaging signals from second pixels 22, but does notperform calculation of the distances to road surface 130 (or in otherwords, roadside region) in roadside boundary regions 140 c, 140 d, 140e, and 140 f by using the TOF distance measuring principle based on theinformation from first a region 110 a, or in other words, the imagingsignals from first pixels 21. Instead, computator 30 calculates thedistances to road surface 130 (or in other words, roadside region) inroadside boundary regions 140 c, 140 d, 140 e, and 140 f based on theinformation from second region 120, or in other words, the imagingsignals from second pixels 22 without using the TOF distance measuringprinciple.

Then, computator 30 calculates the appearance of road surface 130 (or inother words, roadside region) in roadside boundary regions 140 c, 140 d,140 e, and 140 f, and also associates the information from first aregion 110 a with the information from second region 120. Morespecifically, computator 30 makes a comparison between the results ofdetection of road surface 130 (or in other words, roadside region) inroadside boundary regions 140 a and 140 b and the results of detectionof road surface 130 (or in other words, roadside region) in roadsideboundary regions 140 c, 140 d, 140 e, and 140 f. If it is determinedthat road surface 130 (or in other words, roadside region) in roadsideboundary regions 140 a and 140 b and road surface 130 (or in otherwords, roadside region) in roadside boundary regions 140 c, 140 d, 140e, and 140 f are portions of the same road surface 130 (or in otherwords, roadside region) at roadside boundary 140, computator 30associates (correlates) information indicating the distances to roadsurface 130 (or in other words, roadside region) in roadside boundaryregions 140 a and 140 b calculated by using the TOF distance measuringprinciple with the results of detection of road surface 130 (or in otherwords, roadside region) in roadside boundary regions 140 c, 140 d, 140e, and 140 f.

Then, computator 30 estimates the slopes of road surface 130 (or inother words, roadside region) in roadside boundary regions 140 c, 140 d,140 e, and 140 f. More specifically, computator 30 estimates thecontinuity of the roadside shape based on the appearance and the slopesof road surface 130 (or in other words, roadside region) in roadsideboundary regions 140 a and 140 b, and the appearance of road surface 130(or in other words, roadside region) in roadside boundary regions 140 c,140 d, 140 e, and 140 f, and then estimates the slopes of road surface130 (or in other words, roadside region) in roadside boundary regions140 c, 140 d, 140 e, and 140 f based on the estimated continuity of theroadside shape.

By doing so, computator 30 can estimate the slope of road surface 130 ina region other than first a region 110 a with a higher degree ofaccuracy than when calculation is performed based on only the imagingsignals from second pixels 22.

That is, when the subject is an object (roadside boundary 140) that issituated continuously in first region 110 and second region 120, thecomputation result of first region 110 is associated with computation ofsubject information in second region 120.

Alternatively, when the subject is an object (roadside boundary 140)that is situated continuously in first region 110 and second region 120,the computation result of first a region 110 a is associated withcomputation of subject information in a region (second region 120, firstb region 110 b) other than first a region 110 a.

Also, imaging apparatus 1 may be configured to, for example, performimaging using first pixels 21 and imaging using second pixels 22 atdifferent timings at a predetermined frame rate, and perform output ofimaging signals from first pixels 21 (hereinafter also referred to as“IR imaging signals”) and output of imaging signals from second pixels22 (hereinafter also referred to as “W imaging signals”) at differenttimings at the predetermined frame rate.

FIG. 11 is a schematic diagram showing an example of the relationshipbetween the output order of IR imaging signals and the output order of Wimaging signals when imaging apparatus 1 has the above-describedconfiguration.

Furthermore, imaging apparatus 1 may be configured to, for example,interpolate IR imaging signals with W imaging signals.

FIG. 12A is a schematic diagram showing an example of the relationshipbetween the output order of IR imaging signals and the output order ofimaging signals (hereinafter also referred to as “IR interpolationimaging signals”) obtained by interpolating IR imaging signals with Wimaging signals when imaging apparatus 1 has the above-describedconfiguration.

In FIG. 12A, IR interpolation imaging signal a 500 a is an IRinterpolation imaging signal that is generated based on W imaging signala 400 a and used to interpolate between IR imaging signal a 300 a and IRimaging signal b 300 b. IR interpolation imaging signal b 500 b is an IRinterpolation imaging signal that is generated based on W imaging signalb 400 b and used to interpolate between IR imaging signal b 300 b and IRimaging signal c 300 c. IR interpolation imaging signal c 500 c is an IRinterpolation imaging signal that is generated based on W imaging signalc 400 c and used to interpolate between IR imaging signal c 300 c and IRimaging signal d (not shown).

As shown in FIG. 12A, imaging apparatus 1 having the above-describedconfiguration can substantially increase the output frame rate of IRimaging signals. As a result, imaging apparatus 1 having theabove-described configuration can further improve the accuracy ofmeasuring the distance to the subject and/or the accuracy of detectingthe subject.

Also, imaging apparatus 1 may be configured to, for example, generate anIR interpolation imaging signal (for example, IR interpolation imagingsignal a 500 a) based on, in addition to a W imaging signal (forexample, W imaging signal a 400 a) corresponding to the IR interpolationimaging signal, the previous and subsequent IR imaging signals (forexample, IR imaging signal a 300 a and IR imaging signal b 300 b). Withthis configuration, imaging apparatus 1 having the above-describedconfiguration can generate an IR interpolation imaging signal with ahigher degree of accuracy.

Furthermore, imaging apparatus 1 may be configured to, for example,interpolate W imaging signals with IR imaging signals.

FIG. 12B is a schematic diagram showing an example of the relationshipbetween the output order of W imaging signals and the output order ofimaging signals (also referred to as “W interpolation imaging signals”)obtained by interpolating W imaging signals with IR imaging signals whenimaging apparatus 1 has the above-described configuration.

As shown in FIG. 12B, W interpolation imaging signal b 600 b is a Winterpolation imaging signal that is generated based on IR imagingsignal b 300 b and used to interpolate between W imaging signal a 400 aand W imaging signal b 400 b. W interpolation imaging signal c 600 c isa W interpolation imaging signal that is generated based on IR imagingsignal c 300 c and used to interpolate between W imaging signal c 400 band W imaging signal c 400 c.

As shown in FIG. 12B, imaging apparatus 1 having the above-describedconfiguration can substantially increase the output frame rate of Wimaging signals. As a result, imaging apparatus 1 having theabove-described configuration can further improve the accuracy ofmeasuring the distance to the subject and/or the accuracy of detectingthe subject.

Also, imaging apparatus 1 may be configured to, for example, generate aW interpolation imaging signal (for example, W interpolation imagingsignal b 600 b) based on, in addition to an IR imaging signal (forexample, IR imaging signal b 300 b) corresponding to the W interpolationimaging signal, the previous and subsequent W imaging signals (forexample, W imaging signal a 400 a and W imaging signal b 400 b). Withthis configuration, imaging apparatus 1 having the above-describedconfiguration can generate a W interpolation imaging signal with ahigher degree of accuracy.

Other Embodiments

The embodiment given above has been described as an example of atechnique disclosed in the present application. However, the techniqueaccording to the present disclosure is not limited thereto, and is alsoapplicable to embodiments obtained by making modifications,replacements, additions, omissions and the like as appropriate.

(1) In the present disclosure, an example has been described in whichcomputator 30 and controller 40 are implemented by computationprocessing devices such as microprocessors. However, computator 30 andcontroller 40 are not limited to the implementation example given aboveas long as they have the same functions as those of the implementationexample. For example, computator 30 and controller 40 may be configuredsuch that some or all of the structural elements of computator 30 andcontroller 40 are implemented by a dedicated circuit.

(2) The structural elements of imaging apparatus 1 may be configured asindividual single chips by using semiconductor devices such as ICs(Integrated Circuits) or LSIs (Large Scale Integrations), or some or allof them may be configured in a single chip. Also, implementation of anintegrated circuit is not necessarily limited to an LSI, and may beimplemented by a dedicated circuit or a general-purpose processor. It isalso possible to use an FPGA (Field Programmable Gate Array) that can beprogrammed after LSI production or a reconfigurable processor thatenables reconfiguration of the connection and setting of circuit cellsin the LSI. Furthermore, if a technique for implementing an integratedcircuit that can replace LSIs appears by another technique resultingfrom the progress or derivation of semiconductor technology, thefunctional blocks may be integrated by using that technique. Applicationof biotechnology or the like is possible.

(3) Embodiments implemented by any combination of the structuralelements and the functions described in the embodiment given above arealso encompassed in the scope of the present disclosure.

Although only an exemplary embodiment of the present disclosure has beendescribed in detail above, those skilled in the art will readilyappreciate that many modifications are possible in the exemplaryembodiment without materially departing from the novel teachings andadvantages of the present disclosure. Accordingly, all suchmodifications are intended to be included within the scope of thepresent disclosure.

INDUSTRIAL APPLICABILITY

The present disclosure is widely applicable to an imaging apparatus thatimages a subject.

What is claimed is:
 1. An imaging apparatus that is mounted on a vehiclethat runs on a road surface, the imaging apparatus comprising: a lightsource that emits illumination light which is infrared light; asolid-state imaging device that images a subject and outputs an imagingsignal indicating a light exposure amount; and a computator thatcomputes subject information regarding the subject by using the imagingsignal, wherein the solid-state imaging device includes: first pixelsthat image the subject by receiving reflected light that is theillumination light reflected off the subject; and second pixels thatimage the subject by receiving visible light, information indicated byan imaging signal outputted from the first pixels is informationregarding a slope of the road surface, and information indicated by animaging signal outputted from the second pixels is information regardingan appearance of the road surface.
 2. The imaging apparatus according toclaim 1, wherein the information regarding the appearance of the roadsurface is information regarding a roadside boundary.
 3. The imagingapparatus according to claim 1, wherein the first pixels and the secondpixels are placed in alternate rows to form a stripe pattern.
 4. Theimaging apparatus according to claim 1, wherein the imaging apparatusperforms one of forward monitoring and backward monitoring.
 5. Theimaging apparatus according to claim 1, wherein the imaging apparatusperforms one of diagonal monitoring and lateral monitoring, and anillumination angle of the illumination light in a horizontal directionof the vehicle and a viewing angle of an imaging region in thehorizontal direction of the vehicle are substantially equal.
 6. Theimaging apparatus according to claim 1, wherein an imaging region to beimaged by the solid-state imaging device includes a first region that isimaged by at least the first pixels and a second region that is imagedby the second pixels, one of the first region and the second region issituated around the other of the first region and the second region, andthe computator computes the subject information based on informationfrom the first region and information from the second region.
 7. Theimaging apparatus according to claim 6, wherein the subject is an objecton a road surface, the solid-state imaging device successively performsimaging, and when the object is imaged in the first region at a firsttime, and then imaged in the second region at a second time that is timewhen a predetermined period of time has elapsed from the first time,information regarding a distance to the object at the first time is usedto compute the subject information at the second time.
 8. The imagingapparatus according to claim 6, wherein the subject is an object on aroad surface, the solid-state imaging device successively performsimaging, and when the object is imaged in the second region at a firsttime, and then imaged in the first region at a second time that is timewhen a predetermined period of time has elapsed from the first time,information regarding an appearance of the object at the first time isused to compute the subject information at the second time.
 9. Theimaging apparatus according to claim 6, wherein the solid-state imagingdevice successively performs imaging, and the computation performed bythe computator includes associating information from one of the firstregion and the second region at a first time with information from theother of the first region and the second region at a second time that isdifferent from the first time.
 10. The imaging apparatus according toclaim 6, wherein, when the subject is an object that is continuouslysituated in the first region and the second region, a computation resultof the first region is associated with computation of the subjectinformation in the second region.
 11. The imaging apparatus according toclaim 6, wherein the first region includes a first a region wherereflected light that is the illumination light reflected off a roadsurface reaches the solid-state imaging device and a first b regionwhere the reflected light does not reach the solid-state imaging device.12. The imaging apparatus according to claim 11, wherein, when thesubject is an object that is continuously situated in the first regionand the second region, a computation result of the first a region isassociated with computation of the subject information in a region otherthan the first a region.
 13. A solid-state imaging device used in animaging apparatus that is mounted on a vehicle that runs on a roadsurface and includes a light source that emits illumination light whichis infrared light, the solid-state imaging device, and a computator thatcomputes subject information regarding a subject by using an imagingsignal, the solid-state imaging device comprising: first pixels thatimage the subject by receiving reflected light that is the illuminationlight reflected off the subject; and second pixels that image thesubject by receiving visible light, wherein information indicated by animaging signal outputted from the first pixels is information regardinga slope of the road surface, and information indicated by an imagingsignal outputted from the second pixels is information regarding anappearance of the road surface.